Image context aware medical recommendation engine

ABSTRACT

A system (100) for context aware medical recommendations includes a recommendation engine (138) and a user interface (122). The recommendation engine (138) identifies at least one suggested recommendation (140) according to a medical guideline and context in response to a first input indicating a finding (150) in a medical image (112) of a patient. The user interface (122) displays on a display device (120) the at least one suggested recommendation selectable as a second input.

CROSS-REFERENCE TO PRIOR APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.15/767,532 filed Apr. 11, 2028, which is the U.S. National Phaseapplication under 35 U.S.C. § 371 of International Application No.PCT/IB2016/056301, filed on Oct. 20, 2016, which claims the benefit ofU.S. Provisional Patent Application No. 62/248,642, filed on Oct. 30,2015. These applications are hereby incorporated by reference herein.

FIELD OF THE INVENTION

The following generally relates to medical imaging and evaluationguidelines with specific application to healthcare practitioner reviewof electronically accessed medical images.

BACKGROUND OF THE INVENTION

Healthcare practitioners, such as radiologists, are called upon toreview, evaluate, and make recommendations based on medical images ofpatients generated by scanners, such as X-ray Computed Tomography (CT),Magnetic Resonance (MR), Positron Emission Tomography (PET), SingleProton Emission Computed Tomography (SPECT), Ultrasound (US),combinations, and the like. Typically, images are generated by thescanner and stored in a storage system, such as a Picture Archiving andCommunication System (PACS), departmental Radiology Information System(RIS), and the like and/or queued electronically for review by aqualified healthcare practitioner. Diagnostic imaging has seen dramaticincreases in volume. For example, in an analysis of one large healthcareplan, cross section imaging rose from 260 examinations per 1000 planenrollees in 1997 to 478 examinations per 1000 plan enrollees in 2006.

The healthcare practitioner reviews the image, evaluates the image forabnormalities, e.g. positive findings, and if abnormalities are found,typically makes annotations in the image, and then makes arecommendation concerning the patient. The recommendation is included ina report issued concerning the imaging examination, e.g. test results.The recommendation can be guided by a guideline given the context of theevaluation. The context includes a patient context, e.g. patientdemographics, patient history, etc., an image context, e.g. anatomicallocation, type of image, contrast, type of study, etc., and a findingcontext, e.g. lesion, nodule, type of growth, etc.

Few guidelines are mandatory, such as Breast Imaging-Reporting and DataSystem (BI-RADS), which pertains to breast cancer guidelines. Manyguidelines are optional, such as Fleischner Society recommendations forfollow-up of small lung nodules. Some systems approach this with anoptional user selection of a guideline after evaluating an image, whichmay not include a selected guideline. Recommendations can be based onthe training of the healthcare practitioner, which may involverecommendations not based on any guideline or even consider a guideline.

Guidelines are constantly evolving as understandings about diseaseschange, and new guidelines are continuing to emerge. Education forhealthcare practitioners is typically left to radiology departmentsand/or individual practitioners to understand and absorb changes in theguidelines, which can result in applying outdated guidelines or notapplying a guideline. Guidelines provided in the context of a radiologysystem are typically provided as a selection prior to image evaluation.

Even assuming the healthcare practitioner is aware of currentguidelines, identifying contextual information can be time-consuming anderror prone. For example, in the Fleischner guidelines call for thecontextual information which includes an age of the patient, the lungcancer risk factors for the patient, such as smoking, family history,exposure to second hand smoke, radon gas, asbestos, etc., presence orabsence of prior lung nodules for the patient, the number of lungnodules in the current image, and the size of the lung nodules in thecurrent image. Gathering this information typically means that thehealthcare practitioner assembles the information accessing multipledifferent systems, which reduces efficiency and introduces chances oferror. Plain text descriptions written from the assembled informationhave been shown to be frequently inaccurate or incomplete.

SUMMARY OF THE INVENTION

Aspects described herein address the above-referenced problems andothers. The following describes a method and system for a context awarerecommendation concerning a patient, suggested to a healthcarepractitioner reviewing a medical imaging examination. The context awarerecommendation is according to a guideline selected in response to afinding and contextual information of the medical imaging examination.

In one aspect, a system for context aware medical recommendationsincludes a recommendation engine and a user interface. Therecommendation engine identifies at least one suggested recommendationaccording to a medical guideline and context, in response to a firstinput indicating a finding in a medical image of a patient. The userinterface displays on a display device at least one suggestedrecommendation selectable as a second input.

In another aspect, a method of context aware medical recommendations,includes identifying at least one suggested recommendation according toa medical guideline and context information in response to a first inputidentifying a finding in a medical image of a patient, and displaying atleast one suggested recommendation on a display device selectable as asecond input.

In another aspect, a system for context aware medical recommendationsincludes a context unit, a user interface, and a recommendation engine.The context unit determines context and generates a list of possiblefindings based on the context. The user interface displays on a displaydevice the generated list of possible findings as a first input. Therecommendation engine, in response to the first input indicating afinding of the possible findings in the medical image, identifies atleast one suggested recommendation according to a medical guidelineselected from a plurality of medical guidelines according to thedetermined context and the indicated finding. The determined context andthe indicated finding include identification of at least oneabnormality, an anatomical location of the identified at least oneabnormality and at least one quantitative measure of the identified atleast one abnormality. The user interface displays on the display deviceat least one suggested recommendation selectable as a second input.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention may take form in various components and arrangements ofcomponents, and in various steps and arrangements of steps. The drawingsare only for purposes of illustrating the preferred embodiments and arenot to be construed as limiting the invention.

FIG. 1A schematically illustrates an embodiment of a context awarerecommendation engine system.

FIG. 1B schematically illustrates an example suggested recommendation ofthe context aware recommendation engine system in response to a findingand context of a medical imaging examination.

FIG. 2 flowcharts an embodiment of suggesting a contextually awarerecommendation.

DETAILED DESCRIPTION OF EMBODIMENTS

Initially referring to FIGS. 1A and 1B, an embodiment of a context awarerecommendation engine system 100 is schematically illustrated. Acomputing device 110, such as a smartphone, laptop computer, desktopcomputer, tablet, body worn device, and the like, is configured toaccess, retrieve, or receive a medical image 112. The access can belocal or remote. For example, the medical image 112 can be retrievedfrom local memory of the computing device 110, retrieve from a storagesystem 114, such as a Picture Archiving and Communication System (PACS),departmental Radiology Information System (RIS), a web portal, cloudstorage, and the like, or retrieved directly from a scanner 116, such asa CT scanner, MR scanner, PET scanner, SPECT scanner, US scanner, andthe like. The retrieval can include using a network 118, such as theInternet, intranet, public network, private network, combinations, andthe like.

The medical image 112 is displayed on a display device 120 of thecomputing device 110 for review by a healthcare practitioner accordingto a user interface 122. The user interface 122 includes a menu, such asa ring menu 124 with a transparent center, e.g. medical image 112visible through the center, and tool menus located around the ringshape. The tool menus include a measuring tool 126 and a findings tool128. The visible center can be translated to and/or positioned aroundthe region of interest or abnormality, such as a nodule, lesion, and thelike. The healthcare practitioner interacts with the system through theuser interface 122 using an input device 130, such as a touch screen,microphone, mouse, keyboard and the like.

The measuring tool 126, selected with an input, such as a mouse click,screen touch, and the like, generates a quantitative measurement of theabnormality 131, such as a distance measurement, volume measurement,area measurement, volume flow measurement, density measurement, and thelike. For example, using a click and drag function with a mouse, adistance measure of the largest dimension of a lung nodule is made. Themeasuring tool 126 generates a label according to the measurement whichincludes a reference label, e.g. temporary name, such as an alphanumericcharacter, and the quantitative measurement. In the example shown inFIG. 1B, the label includes “A:” and “5.9 mm.” The user interface caninclude displaying the information used to generate the measurement,such as a geometric shape indicative of the measured dimensions, e.g. acontrasted line, such as a dotted line for a distance measurement, acolor and/or patterned contrasted area for an area measurement, and thelike.

A context unit 132 determines context of the medical image 112. Thecontext includes patient context, study context, and/or finding context.The patient context can include patient demographics, such as age, riskfactors, and/or prior findings. For example, age can be obtained frommetadata of the medical image 112, such as a DICOM header. Priorfindings can be obtained from natural language processing (NLP) ofreports 133 generated previously for the patient using NLP techniquesknown in the art. Study context can include a reason for the medicalimaging examination, e.g. from the metadata and/or physician orderrequesting the medical imaging examination, relevant prior studies, e.g.prior medical images of the patient in a storage system 114 and/orreports 133 previously generated for the patient, imaging modality andanatomical location, e.g. obtained from medical image metadata. Thefinding context can include identification of the abnormality or findingtype, e.g. nodule, measurement sizes, anatomical locations, imagenumbers and/or series types. For example, the context unit 132determines the age of the patient from a DICOM header of the currentmedical image, determines a history of smoking and prior lung nodulesfrom NLP of prior reports of the patient, and a nodule in the lung andsize measurement corresponding to inputs from user interface 122.

The findings tool 128 receives an input indicative of the finding 134.The findings tool 128 can generate a list of possible finding 136 basedon context from the context unit 132. For example, in the medical image112 of a chest, the possible findings can be limited to those based onthe anatomical location of the image, e.g. abnormalities of the chest,and/or based on the location of the ring menu 124 relative to a morespecific anatomical location within the image, such as possible findingswithin a lower right lobe of the lung, e.g. displayed finding includesthe anatomical location and identification of abnormality, e.g. nodule(undifferentiated), metastatic tumor, benign tumor. In one embodiment,the findings tool 128 includes the context of the measurement, which caninclude differences in measurements from prior medical imaging studies.For example, the context unit identifies a corresponding nodule in aprior imaging study of the patient, e.g. image registration and/or userinput, and a change in size between the prior imaging study and thecurrent imaging study is computed. Based on the change in size, thepossible findings can be further limited, e.g. increase greater than athreshold amount is a lesion. In one embodiment, the list of possiblefinding types 136 is dynamically adjusted by the user interface 122. Forexample, as inputs are received, such as individual characters input viathe input device 130, the list of possible finding 136 is character bycharacter matched with the input to reduce displayed the possiblefinding 136.

A recommendation engine 138, in response to an input indicating thefinding identifies a suggested recommendation 140 according to a medicalguideline 142 and the context. The finding 134 with the context includesthe identity or type of abnormality, the anatomical location and thequantified measurement 131. The recommendation engine 138 selects themedical guideline from a data store of medical guidelines 144 accordingto the context. For example, if the context includes the anatomicallocation of a breast, and a finding of a nodule or lesion, then theguideline selected is BI-RADS. In another example, if the anatomicallocation is the lung, and the finding is an incidental nodule, then theguideline selected is a Fleischner.

The data store of guidelines 144 can include mandatory and non-mandatoryor optional guidelines. The data store of guidelines 144 can includecomputer storage, such as local or remote storage, cloud storage,distributed storage, and the like. The data store of guidelines 144 caninclude system, file, and/or database organization. The data store ofguidelines 144 can include optimized access according to findings 134including anatomical location and/or quantitative measurements 131. Thedata store of guidelines 144 can include optimized access according toother contextual information, such as patient demographics, type ofimaging examination, risk factors, and the like.

The suggested recommendation 140 can include multiple suggestedrecommendations. In some instances this may be due to partial contextinformation. For example, where the risk factors are determinable fromthe available information for a lung nodule of 5.9 mm, a suggestedrecommendation for a high risk patient and a second suggestedrecommendation for a low risk patient are displayed. In some instances,this may be due to different guidelines for the same finding andcontext.

The user interface 122 displays the suggested recommendation 140 orrecommendations, which can be displayed as a selectable menu item. Forexample, in FIG. 1B, the selectable menu item is shown as a cascadingdrop down box. The displayed suggested recommendation 140 can include anidentity of the guideline 142, e.g. Fleischner, BI-RADS, etc. Thedisplayed suggested recommendation 140 can include a rule 146 which mapsthe finding 134 and context to the suggested recommendation 140. Forexample, a rule of finding type of lung nodule of 4-6 mm and low riskfactors maps to a suggested Fleischner recommendation of “Follow-up CTat 12 months; if unchanged, no further follow-up.”

The suggested recommendation 140 is selected in response to an input,such as a mouse click, voice command, screen touch change, and the like.The user interface 122 can include a response 148 which assembles thefindings 150 and the selected recommendation 152. The response 148 caninclude other contextual information, such as a prior medical imagingexamination 154, e.g. used for comparative measurements. The userinterface 122 can generate a report of the imaging examination studyincluding the displayed medical image 112. The generated report can bestored in the reports 133 and/or distributed electronically.

The user interface 122, the context unit 132, and the recommendationengine 138 comprise one or more configured processors 156, e.g., amicroprocessor, a central processing unit, a digital processor, and thelike) are configured to execute at least one computer readableinstruction stored in a computer readable storage medium, which excludestransitory medium and includes physical memory and/or othernon-transitory medium. The processor 156 may also execute one or morecomputer readable instructions carried by a carrier wave, a signal orother transitory medium. The processor 156 can include local memoryand/or distributed memory. The processor 156 can includehardware/software for wired and/or wireless communications. For example,the lines indicate communications paths between the various componentswhich can be wired or wireless. The processor 156 can comprise thecomputing device 110.

With reference to FIG. 2 , an embodiment of suggesting a contextuallyaware recommendation is flowcharted. At 200, an identified abnormalityin a displayed medical image 112 of a patient can be measured. Themeasuring can include a distance, an area, a volume, a rate, a density,combinations, and the like of aspects of the abnormality. Themeasurement can be received from an input and/or determined from themedical image based on the input.

At 202, context is determined, which can include patient context, imagecontext and/or finding context. The context can be determined frominformation stored in the metadata of the medical image 112, priorimages and/or prior examinations of the patient, and/or direct entry.

At 204, a finding is identified. The finding includes a type ofabnormality in the medical image 112. The finding includes themeasurement. The identification of a finding can be in response to aninput indicative of the abnormality selected from a list of possiblefindings. The displayed list of possible findings can be limited by thecontext, e.g. findings possible according to the anatomical location,imaging modality, type of imaging examination, measurement, and/ormeasurement type, and the like.

In response to an input selecting the finding or inputting the finding,one or more suggested recommendations for the patient are displayed at206. The one or more suggested recommendations are according to one ormore medical guidelines selected according to the finding and thecontext. The medical guidelines can include mandatory and/ornon-mandatory guidelines. The suggested recommendations are selectedaccording to a rule which maps the finding and the context to aguideline or guidelines and suggested recommendations within aguideline. The displayed suggested recommendation can includeidentification of the guideline. The displayed suggested recommendationcan include the rule used to determine the suggested recommendation. Thedisplayed suggested recommendations can include partial context, whichsatisfies only part of the rule.

At 208, an input selects one of the displayed suggested recommendationsfor the patient as a recommendation. At 210, a response is assembledwhich includes the recommendation and the findings. The response caninclude information determined from the context, such as prior imagingexaminations, specific images, prior measurements, and/or determinedrisk factors and their sources. A report can be generated from theassembled response, which in some instances is a result of reading theimaging examination. The response can be assembled as each portion isobtained. For example, as context information is identified, such asprior imaging examinations, reference information can be included in thedisplay. As each finding is made, the assembled display is updated. Theassembled display is updated as the recommendation is selected from thesuggested recommendations.

The invention has been described with reference to the preferredembodiments. Modifications and alterations may occur to others uponreading and understanding the preceding detailed description. It isintended that the invention be constructed as including all suchmodifications and alterations insofar as they come within the scope ofthe appended claims or the equivalents thereof.

1. A system for context aware medical recommendations, comprising: adisplay device configured to display a user-identified medical image ofa subject and a user-positionable menu over a sub-portion of theuser-identified medical image, wherein the user-positionable menuincludes a findings tool and a measurement tool; an input deviceconfigured to receive a first user input identifying a location of afinding of interest in a region of interest within the sub-portion withthe measurement tool while the menu is displayed over the sub-portionand invoking the measurement tool to take a measurement of the findingof interest; a context unit configured to determine a context includingat least one of a context of the subject, a context of an imaging studyproducing the displayed user-identified medical image including ananatomical region of the sub-portion and a context of the findingincluding the measurement and an anatomical location of the finding;wherein the input device is further configured to receive a second userinput identifying a type of the finding from a displayed list of typesof possible findings for the region of interest with the findings tool,wherein the findings tool generates the list based on the context; and arecommendation engine configured to identify multiple recommendedactions for the finding based on a medical guideline for the type of thefinding and the context, and display the multiple recommended actions ina recommendation window.
 2. The system according to claim 1, wherein thefindings include identification of at least one abnormality, ananatomical location of the identified at least one abnormality and atleast one quantitative measure of the identified at least oneabnormality.
 3. The system according to claim 1, wherein therecommendation engine identifies the medical guideline from a pluralityof medical guidelines based on the context, the identified medicalguideline indicates a follow up action for the type of the finding forthe region of interest, and the recommended action includes the followup action from the identified medical guideline.
 4. The system accordingto claim 3, wherein the follow up action is an examination within apredetermined time period.
 5. The system according to claim 1, whereineach of the multiple recommended actions includes an identification ofthe guideline and a rule which maps the finding and the context to therespective recommended action.
 6. A method of context aware medicalrecommendations, comprising: displaying a medical image of a subject;superimposing a user-positionable menu over a sub-portion of thedisplayed user-identified medical image, wherein the user-positionablemenu includes a findings tool and a measurement tool; receiving an inputthat invokes the measurement tool to generate a measurement of tissue ofinterest in the sub-portion of the medical image; identifying ananatomical location of the tissue of interest from the anatomy in themedical image based on a user input received while the user-positionablemenu is displayed over the sub-portion; determining a context for theexamination based on the measurement and the anatomical location;displaying a list of possible types of findings based on the context;receiving a first input identifying a type of a finding from thedisplayed list of types of possible findings; displaying a list ofpredetermined recommended actions for the finding in a recommendationwindow based on the type of the finding, a medical guideline for thetype of the finding, and the context; and receiving a second inputidentifying at least one recommended action from the list of thepredetermined recommended actions.
 7. The method according to claim 6,wherein the findings include identification of at least one abnormality,an anatomical location of the identified at least one abnormality and atleast one quantitative measure of the identified at least oneabnormality.
 8. The method according to claim 6, further includingidentifying the medical guideline, wherein identifying the medicalguideline includes selecting a medical guideline from a plurality ofmedical guidelines based on the context, and the plurality of medicalguidelines includes at least the Fleischner Society medical guidelineand the Breast Imaging-Reporting and Data System medical guideline. 9.The method according to claim 6, wherein the identified medicalguideline is a non-mandatory medical guideline or optional medicalguideline.
 10. The method according to claim 6, wherein determining thesecond context of the medical image includes using information retrievedfrom at least one of: metadata corresponding to the medical image orpatient information obtained from natural language processing of priorreports about the patient.
 11. The method according to claim 6, whereinthe at least one recommended action includes limiting a plurality ofrecommended actions according to the context.
 12. The method accordingto claim 6, wherein identifying the guideline includes mapping thefinding and the context to the suggested at least one recommended actionaccording to a rule.
 13. The method according to claim 6, wherein theuser-positionable menu includes a transparent sub-region, and a portionof the displayed user-identified medical image behind theuser-positionable menu is visible through the transparent sub-region,and further including: positioning the user-positionable menu about aregion of interest in the displayed user-identified medical image suchthat the region of interest is visible through the transparentsub-region and based on a user input.
 14. The method according to claim13, wherein the user-positionable menu includes an opaque ring shapedregion surrounding the transparent sub-region.
 15. The method accordingto claim 14, wherein the findings tool and the measurement tool arelocated at different arc segments of the user-positionable menu.
 16. Themethod according to claim 11, further including: taking a quantitativegeometric measurement of the finding in the region of interest with themeasurement tool; displaying a label identifying a type of thequantitative geometric measurement and a numerical value of thequantitative geometric measurement in connection with the finding; anddisplaying the summary window with the type of finding, the userselected sub-set of the list of the predetermined recommended actions,and the value of the measurement.
 17. A non-transitory computer readablemedium encoded with computer executable instructions, which, whenexecuted by a processor of a computing system, causing the processor to:display a user-identified medical image of a subject; superimpose auser-positionable menu over a sub-portion of the displayeduser-identified medical image, wherein the user-positionable menuincludes a findings tool configured to identify a type of a finding, ameasurement tool configured to take a measurement of the finding and atransparent sub-region, and further including: position theuser-positionable menu about a region of interest in the displayeduser-identified medical image such that the region of interest isvisible through the transparent sub-region and based on a user input;identify a location of a finding of interest in the region of interestwith the measurement tool based on a user input received while theuser-positionable menu is displayed over the sub-portion; receive aninput identifying the type of a finding from a displayed list of typesof possible findings for the region of interest with the finding tool;determine a context including at least one of a context of the subject,a context of an imaging study producing the displayed user-identifiedmedical image, and a context of the finding; identify a list ofpredetermined recommended actions for the finding according to the typeof the finding, a medical guideline for the type of the finding, and theat least one context, wherein the medical guideline includes ameasurement range and a risk for the type of the finding; display thelist of predetermined recommended actions in a recommendation window;and receive an input identifying at least one recommended action fromthe list of the predetermined recommended actions in the recommendationwindow.
 18. The non-transitory computer readable medium encoded of claim7, wherein the computer executable instructions further cause theprocessor to: take a quantitative geometric measurement of the findingin the region of interest with the measurement tool; display a labelidentifying a type of the quantitative geometric measurement and anumerical value of the quantitative geometric measurement in connectionwith the finding; and display the value of the measurement in atransparent sub-region of the user-positionable menu, which issurrounded by an opaque ring shaped region of the user-positionablemenu.
 19. The system according to claim 1, wherein the summary windowfurther includes an identification of a prior medical imagingexamination used for comparative measurements.
 20. The system accordingto claim 1, wherein the user selected recommended action in the summarywindow is the only recommended action in the summary window.